Skip to content

Fix auto-recovery behavior for AMQ119002 producer failures#564

Merged
Havret merged 6 commits into
masterfrom
fix/auto-recovery-amq119002-producer
May 4, 2026
Merged

Fix auto-recovery behavior for AMQ119002 producer failures#564
Havret merged 6 commits into
masterfrom
fix/auto-recovery-amq119002-producer

Conversation

@Havret
Copy link
Copy Markdown
Owner

@Havret Havret commented Apr 30, 2026

Fixes #562

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adjusts auto-recovery behavior for producers when the broker closes a producer link with terminal AMQP error conditions (notably AMQ119002 / amqp:not-found), preventing repeated recovery attempts that can flood logs.

Changes:

  • Treat ProducerClosedException with ErrorCode.NotFound (and UnauthorizedAccess) as terminal for AutoRecoveringProducer by terminating the producer instead of retrying/recovering.
  • Treat UnauthorizedAccess as terminal for AutoRecoveringAnonymousProducer by terminating the anonymous producer instead of retrying.
  • Add unit tests covering terminal close behavior for producer send/send-async and anonymous producer send/send-async.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
test/ArtemisNetClient.UnitTests/AutoRecovering/AutoRecoveringProducerSpec.cs Adds tests asserting producers terminate (no further retries) when the producer link is closed with amqp:not-found (AMQ119002).
test/ArtemisNetClient.UnitTests/AutoRecovering/AutoRecoveringAnonymousProducerSpec.cs New tests asserting anonymous producers terminate when link is closed with amqp:unauthorized-access.
src/ArtemisNetClient/AutoRecovering/AutoRecoveringProducer.cs Terminates the auto-recovering producer for terminal broker-side close reasons (UnauthorizedAccess, NotFound) instead of retrying.
src/ArtemisNetClient/AutoRecovering/AutoRecoveringAnonymousProducer.cs Terminates the auto-recovering anonymous producer on UnauthorizedAccess instead of retrying.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +250 to +252
var host = CreateOpenedContainerHost();
var linkProcessor = host.CreateTestLinkProcessor();

Copy link

Copilot AI Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CreateOpenedContainerHost() opens a listening TestContainerHost, but this test doesn't dispose it. Please ensure host is disposed (e.g., using var host = ...) to avoid leaking resources/ports between tests.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot apply changes based on this feedback

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed both test methods (Should_not_retry_SendAsync_when_producer_link_closed_with_not_found and Should_not_retry_Send_when_producer_link_closed_with_not_found) to use using var host = CreateOpenedContainerHost() for proper disposal. Commit: b25ba1c

[Fact]
public async Task Should_not_retry_SendAsync_when_producer_link_closed_with_not_found()
{
var host = CreateOpenedContainerHost();
Copy link

Copilot AI Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CreateOpenedContainerHost() opens a listening TestContainerHost, but this test never disposes it. Please wrap host in using (or dispose via DisposeUtil) to avoid leaking listeners across the test run.

Suggested change
var host = CreateOpenedContainerHost();
using var host = CreateOpenedContainerHost();

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +33 to 35
catch (ProducerClosedException e) when (IsTerminalProducerException(e))
{
await TerminateAsync(e).ConfigureAwait(false);
Comment on lines +32 to +34
catch (ProducerClosedException e) when (e.ErrorCode is ErrorCode.UnauthorizedAccess)
{
await TerminateAsync(e).ConfigureAwait(false);
Comment on lines +67 to +69
catch (ProducerClosedException e) when (IsTerminalProducerException(e))
{
TerminateAsync(e).GetAwaiter().GetResult();
}
catch (ProducerClosedException e) when (e.ErrorCode is ErrorCode.UnauthorizedAccess)
{
TerminateAsync(e).GetAwaiter().GetResult();
@Havret
Copy link
Copy Markdown
Owner Author

Havret commented May 2, 2026

@copilot apply changes based on the comments in this thread

Copy link
Copy Markdown
Contributor

Copilot AI commented May 2, 2026

@copilot apply changes based on the comments in this thread

Fixed in commit 35efb24. AutoRecoveringProducerBase.TerminateAsync now fires Closed?.Invoke(this) after disposing the underlying resource. This triggers OnRecoverableClosed in AutoRecoveringConnection, which unsubscribes event handlers and removes the producer from _recoverables — so it won't be recreated on subsequent connection recovery cycles.

Added tests for both the regular producer (Should_not_recreate_producer_on_connection_recovery_after_terminal_not_found_error) and the anonymous producer (Should_not_recreate_anonymous_producer_on_connection_recovery_after_terminal_unauthorized_access_error) to verify this behavior.

Copilot finished work on behalf of Havret May 2, 2026 12:17
@Havret Havret force-pushed the fix/auto-recovery-amq119002-producer branch from 35efb24 to a61042b Compare May 3, 2026 10:04
@Havret
Copy link
Copy Markdown
Owner Author

Havret commented May 3, 2026

@copilot apply changes based on the comments in this thread

Copy link
Copy Markdown
Contributor

Copilot AI commented May 3, 2026

@copilot apply changes based on the comments in this thread

That change was already applied in commit 0a2905a — both Should_not_retry_SendAsync_when_producer_link_closed_with_not_found and Should_not_retry_Send_when_producer_link_closed_with_not_found already use using var host = CreateOpenedContainerHost() for proper disposal.

Copilot finished work on behalf of Havret May 3, 2026 10:05
@Havret Havret force-pushed the fix/auto-recovery-amq119002-producer branch from b8102a3 to 47261c6 Compare May 4, 2026 17:05
@Havret Havret force-pushed the fix/auto-recovery-amq119002-producer branch from 47261c6 to 0f12d8f Compare May 4, 2026 17:42
@Havret Havret merged commit 1404aa7 into master May 4, 2026
1 check passed
@Havret Havret deleted the fix/auto-recovery-amq119002-producer branch May 4, 2026 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AutoRecoveringConnection repeatedly logs "Producer suspended/resumed" without recovering

3 participants